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Preface 


This  report  is  a  compilation  of  papers  by  the  PATR  group  in 
the  Artificial  Intelligence  Center  at  SRI  International  reporting  on 
ongoing  research  on  both  practical  and  theoretical  issues  concern¬ 
ing  grammar  formalisms.  The  current  formalism  being  simultane¬ 
ously  designed,  implemented  and  used  by  the  group,  PATR-II,  is 
based  on  unification  of  directed-graph  structures.  Unification  is 
thus  a  theme  both  of  our  research,  and  of  the  papers  reproduced 
in  this  volume.  These  papers  provide  an  overview  of  the  design  of 
PATR-II  (Chapter  1),  a  discussion  of  the  use  of  disjunction  and 
negation  in  unification-based  feature  systems  (Chapter  2),  and  a 
theoretical  framework  for  unification-based  grammar  formalisms 
which  is  founded  on  the  domain-theoretic  techniques  of  Dana  Scott 
(Chapter  3).  All  three  chapters  are  versions  of  papers  presented  at 
the  Tenth  International  Conference  on  Computational  Linguistics 
at  Stanford  University,  Stanford,  California  during  July  2  through 
7,  1984. 

The  research  was  initially  part  of  the  KLAUS  (Knowledge 
Learning  And  Using  System)  project  at  SRI,  set  up  with  the  inten¬ 
tion  of  experimenting  with  mathematically  well-founded  alterna¬ 
tives  to  the  DIALOGIC  natural-language  processing  system.  The 
more  theoretical  research  was  made  possible  in  part  by  a  gift  from 
the  Systems  Development  Foundation  and  was  conducted  as  part 
of  a  coordinated  research  effort  with  the  Computer  Languages  sub¬ 
program  at  the  Center  for  the  Study  of  Language  and  Information, 
Stanford  University. 

The  PATR  group  at  SRI  is  a  rather  liquid  group  of  researchers 
which  has  included,  at  various  times,  John  Bear,  Lauri  Kart- 
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tunen,  Fernando  Pereira,  Jane  Robinson,  Stan  Rosenschein,  Stuart 
Shieber,  Susan  Stucky,  Mabry  Tyson,  and  Hans  Uszkoreit.  The  re¬ 
search  reported  here  is  a  direct  result  of  their  aid  and  interaction. 
However,  they  should  not  be  blamed  for  any  errors  in  the  present 
work,  nor  should  the  opinions  expressed  herein  be  construed  as 
indicative  of  their  personal  predilections. 
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Chapter  1. 

The  Design  of  a  Computer 
Language  for  Linguistic 
Information 


This  chapter  was  written  by  Stuart  M.  Shieber  of  the  Artificial 
Intelligence  Center,  SRI  International  and  the  Center  for  the  Study 
of  Language  and  Information,  Stanford  University. 


Abstract^ 

A  considerable  body  of  accumulated  knowledge  about  the  de¬ 
sign  of  languages  for  communicating  information  to  computers  has 
been  derived  from  the  subfields  of  programming  language  design 
and  semantics.  It  has  been  the  goal  of  the  PATR  group  at  SRI  to 

'This  research  has  been  made  possible  in  part  by  a  gift  from  the  Systems  Devel¬ 
opment  Foundation,  and  was  also  supported  by  the  Defense  Advanced  Research 
Projects  Agency  under  Contracts  N00039-80-C-0575  and  N00039-84-K-0078 
with  the  Naval  Electronic  Systems  Command.  The  views  and  conclusions  con¬ 
tained  in  this  document  are  those  of  the  author  and  should  not  be  interpreted 
as  representative  of  the  official  policies,  either  expressed  or  implied,  of  the  De¬ 
fense  Advanced  Research  Projects  Agency,  or  the  United  States  government. 

The  author  is  indebted  to  Fernando  Pereira,  Barbara  Grosz,  and  Ray  Perrault 
for  their  comments  on  earlier  drafts. 
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utilize  a  relevant  portion  of  this  knowledge  in  implementing  tools 
to  facilitate  communication  of  linguistic  information  to  computers. 
The  PATR-II  formalism  is  our  current  computer  language  for  en¬ 
coding  linguistic  information.  This  paper,  a  brief  overview  of  that 
formalism,  attempts  to  explicate  our  design  decisions  in  terms  of 
a  set  of  properties  that  effective  computer  languages  should  incor¬ 
porate. 


1.1.  Introduction 

The  goal  of  natural-language  processing  research  can  be  stated 
quite  simply:  to  endow  computers  with  human  language  capability. 
The  pursuit  of  this  objective,  however,  has  been  a  difficult  task  for 
at  least  two  reasons:  first,  this  capability  is  far  from  being  a  well- 
understood  phenomenon;  second,  the  tools  for  teaching  computers 
what  we  do  know  about  human  language  are  still  very  primitive. 
The  solution  of  these  problems  lies  within  the  respective  domains 
of  linguistics  and  computer  science. 

Similar  problems  have  arisen  previously  in  computer  science. 
Whenever  a  new  computer  application  area  emerges,  there  follow 
new  modes  of  communication  with  computers  that  are  geared  to¬ 
wards  such  areas.  Computer  languages  are  a  direct  result  of  this 
need  for  effective  communication  with  computers.  A  considerable 
body  of  accumulated  knowledge  about  the  design  of  languages  for 
communicating  information  to  computers  has  been  derived  from 
.the  subfields  of  programming  language  design  and  semantics.  It 
has  been  the  goal  of  the  PATR  group  at  SRP  to  utilize  a  rele¬ 
vant  portion  of  this  knowledge  in  implementing  tools  to  facilitate 
communication  of  linguistic  information  to  computers. 

The  PATR-II  formalism  is  our  current  computer  language  for 
encoding  linguistic  information.  This  paper,  a  brief  overview 

^This  rather  liquid  group  has  included  at  various  times;  John  Bear,  Lauri  Kart- 
tunen,  Fernando  Pereira,  Jane  Robinson,  Stan  Rosenschein,  Susan  Stucky, 
Mabry  Tyson,  Hans  Uszkoreit,  and  the  author. 
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of  that  formalism,  attempts  to  explicate  our  design  decisions  in 
terms  of  a  set  of  properties  that  effective  computer  languages 
should  incorporate,  namely;  simplicity,  power,  mathematical  well- 
foundedness,  flexibility,  implementability,  modularity,  and  declar¬ 
ativeness.  More  extensive  discussions  of  various  aspects  of  the 
PATR-II  formalism  and  systems  can  be  found  in  papers  by  Shieber 
et  ai,  [83],  Pereira  and  Shieber  [84]  and  Karttunen  [84], 

The  notion  of  designing  specialized  computer  languages  and 
systems  to  encode  linguistic  information  is  not  new;  PROGRAM¬ 
MAR  [Winograd,  72],  ATNs  [Woods,  70],  and  DIALOGIC  [Grosz, 
el  ai,  82]  are  but  a  few  of  the  better-known  examples.  Further¬ 
more,  a  trend  has  arisen  recently  in  linguistics  towards  declara¬ 
tiveness  in  grammar  formalisms — for  instance,  lexical-functional 
grammar  (LFG)  [Bresnan,  83],  generalized  phrase-structure  gram¬ 
mar  (GPSG)  [Gazdar  and  Pullum,  82]  and  functional  unification 
grammar  (UG)  [Kay,  83].  Finally,  in  computer  science  there  has 
been  a  great  deal  of  interest  in  declarative  languages  (e.g.,  logic 
programming  and  specification  languages),  and  their  supporting 
denotational  semantics.  But  to  our  knowledge,  no  attempt  has  yet 
been  made  to  combine  the  three  approaches  so  as  to  yield  a  declar¬ 
ative  computer  language  with  clear  semantics  designed  specifically 
for  encoding  linguistic  information.  Such  a  language,  of  which 
PATR-II  is  an  example,  would  reflect  a  felicitous  convergence  of 
ideas  from  linguistics,  artificial  intelligence,  and  computer  science. 


1.2.  The  Critical  Properties  of  the  Lan¬ 
guage 

It  is  not  the  purpose  of  this  paper  to  provide  a  comprehensive 
description  of  the  PATR-II  project,  or  even  of  the  formalism  itself. 
Rather,  we  will  discuss  briefly  the  critical  properties  of  PATR- 
II  to  give  a  flavor  for  our  approach  to  the  design  of  the  language. 
References  to  papers  with  more  complete  descriptions  of  particular 
aspects  of  the  project  are  provided  when  appropriate. 
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1,2.1.  Simplicity:  An  Introduction  to  the  PATR- 
II  Formalism 

Building  on  a  convergence  of  ideas  from  the  linguistics  and 
A1  communities,  PATR-II  takes  as  its  primitive  operation  an  ex¬ 
tended  pattern-matching  technique,  unification,  first  used  in  logic 
and  theorem-proving  research  and  lately  finding  its  way  into  re¬ 
search  in  linguistics  [Kay,  79;  Gazdar  and  Pullum,  82]  and  knowl¬ 
edge  representation  [Reynolds,  70;  Ait-Kaci,  83].  Instead  of  uni¬ 
fying  logic  terms,  however,  PATR  unification  operates  on  directed 
acyclic  graphs  (DAG).* 

DAGs  can  be  atomic  symbols  or  sets  of  label/value  pairs,  where 
the  values  are  themselves  DAGs  (either  atomic  or  complex).  Two 
labels  can  have  the  same  value — thus  the  use  of  the  term  graph 
rather  than  tree.  DAGs  are  notated  either  by  drawing  the  graph 
structure  itself,  with  the  labels  marking  the  arcs,  or,  as  in  this 
paper,  by  notating  the  sets  of  label/value  pairs  in  square  brackets, 
with  the  labels  separated  from  their  values  by  a  colon;  e.g.,  a  DAG 
associated  with  the  verb  “kniglit”  (as  in  “Uther  wants  to  knight 
Arthur”)  would  appear  (in  at  least  one  of  our  grammars)  as 

[cat :  V 

head:  [aux:  false 

form:  nonflnite 
voice;  active 
trans;  [pred;  knight 
argl:  <fll34> 

[] 

arg2:  <fll3B> 

[]]] 

_ syncat:  [first:  [cat:  np 

^Technically,  these  are  rooted,  directed,  acyclic  graphs  with  labeled  arcs.  Formal 
definition  of  these  and  other  technical  notions  can  be  found  in  Appendix  A  of 
Shieber  et  al.  [83].  Note  that  some  implementations  have  been  extended  to 
handle  cyclic  graph  structures  as  well  as  graph  structures  with  disjunction  and 
negation  [Karttunen,  84j. 
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head:  [trans:  <11134>]] 
rest:  [first:  [cat:  np 

head:  [trans:  <fil38>]] 
rest:  <fii40> 
lambda] 

tail:  <fil40>]] 

Reentrant  structure  is  notated  by  labeling  the  DAG  with  an  ar¬ 
bitrary  label  (in  angle  brackets),  then  using  that  label  for  future 
references  to  the  DAG. 

Associated  with  each  entry  in  the  lexicon  is  a  set  of  DAGs.'* 
The  root  of  each  DAG  will  have  an  arc  labeled  cal  whose  value 
will  be  the  category  of  the  associated  lexical  entry.  Other  arcs 
may  encode  information  about  the  syntactic  features,  translation, 
or  syntactic  subcategorization  of  the  entry.  But  only  the  label  cat 
has  any  special  significance;  it  provides  the  link  between  context- 
free  phrase  structure  rules  and  the  DAGs,  as  explicated  below. 

PATR-II  grammars  consist  of  rules  with  a  context-free  phrase 
structure  portion  and  a  set  of  unifications  on  the  DAGs  associated 
with  the  constituents  that  participate  in  the  application  of  the  rule. 
The  grammar  rules  describe  how  constituents  can  be  built  up  to 
form  new  constituents  with  associated  DAGs.  The  right  side  of 
the  rule  lists  the  cal  values  of  the  DAGs  associated  with  the  filial 
constituents;  the  left  side,  the  cal  of  the  parent.  The  associated 
unifications  specify  equivalences  that  must  exist  among  the  various 
DAGs  and  sub-DAGs  of  the  parent  and  children.  Thus,  the  for¬ 
malism  uses  only  one  representation — DAGs — for  lexical,  syntac¬ 
tic,  and  semantic  information,  and  one  operation — unification — on 
this  representation. 

By  way  of  example,  we  present  a  trivial  grammar  for  a  fragment 
of  English  with  a  lexicon  associating  words  with  DAGs. 

_ S  NP  VP 

'*In  our  impiementation,  this  association  is  not  directly  encoded — since  this 
would  yield  a  grossly  inefficient  characterization  of  the  lexicon — but  is  mediated 
by  a  morphological  analyzer.  See  Section  1.2.6  for  further  details. 
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<  VP  agr> 

= 

<NP  agr> 

VP  V  NP 

<  VP  agr> 

= 

<  V  agr> 

Uther: 

<cat> 

=z 

np 

<agr  nutnber> 

singular 

<agr  person> 

— 

third 

Arthur: 

<cat> 

= 

np 

<agr  number> 

= 

singular 

<agr  person> 

= 

third 

knights: 

<cat> 

=: 

V 

<agr  number> 

= 

singular 

<agr  person> 

third 

This  grammar  (plus  lexicon)  admits  the  two  sentences  “Uther 
knights  Arthur”  and  “Arthur  knights  Uther.”  The  phrase  struc¬ 
ture  associated  with  the  first  of  these  is: 

[s  [np  Uther]  [vp  [v  knights]  [np  Arthur]]] 

The  VP  rule  requires  that  the  agr  feature  of  the  DAG  associ¬ 
ated  with  the  VP  be  the  same  as  (unified  with)  the  agr  of  the  V. 
Thus,  the  VP’s  agr  feature  will  have  as  its  value  the  same  node  as 
the  V’s  agr,  and  hence  the  same  values  for  the  person  and  num- 
6er  features.  Similarly,  by  virtue  of  the  unification  associated  with 
the  S  rule,  the  NP  will  have  the  same  agr  value  as  the  VP  and, 
consequently,  the  V.  We  have  thus  encoded  a  form  of  subject-verb 
agreement. 

Note  that  the  process  of  unification  is  order-independent.  For 
instance,  we  would  get  the  same  effect  regardless  of  whether  the 
unifications  at  the  top  of  the  parse  tree  were  effected  before  or  after 
those  at  the  bottom.  In  either  case,  the  DAG  associated  with,  e.g., 
the  VP  node  would  be 
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[cat:  vp 

agr;  [person:  third 

number:  singular]] 

These  trivial  examples  of  grammars  and  lexicons  offer  but 
a  glimpse  of  the  techniques  used  in  writing  PATR-II  grammars, 
and  do  not  begin  to  employ  the  power  of  unification  as  a  general 
information-passing  mechanism.  Examples  of  the  use  of  PATR- 
II  for  encoding  much  more  complex  linguistic  phenomena  can  be 
found  in  Shieber  et  al.  [83]. 

1.2.2.  Power;  Two  Variants 

Augmented  phrase-structure  grammars  such  as  PATR-II  can  in 
fact  be  quite  powerful.  The  ability  to  encode  unbounded  amounts 
of  information  in  the  augmentations  (which  PATR-II  obviously  al¬ 
lows)  gives  this  formalism  the  power  of  a  Turing  machine.  As  a 
linguistic  theory,  this  much  power  might  be  considered  disadvan¬ 
tageous;  as  a  computer  language,  however,  such  power  is  clearly 
desirable,  since  the  intent  of  the  language  is  to  enable  the  modeling 
of  many  kinds  of  linguistic  analyses  from  a  range  of  theories.  As 
such,  PATR-II  is  a  tool,  not  a  result. 

Nevertheless,  a  good  case  could  be  made  for  maintaining  at 
least  the  decidability  of  determining  whether  a  string  is  admitted 
by  a  PATR-II  grammar.  This  property  can  be  ensured  by  requiring 
the  context-free  skeleton  to  have  the  property  of  off-line  paraabilxty 
[Pereira,  83],  which  was  used  originally  in  the  definition  of  LFG  to 
maintain  the  decidability  of  that  formalism  [Kaplan  and  Bresnan, 
83].  Off-line  parsability  requires  that  the  context-free  “skeleton” 
of  the  grammar  allows  no  trivial  cyclic  derivations  of  the  form 
A  =5>  A. 


1.2.3.  Mathematical  Well~Foundedness:  A  De- 
notatlonal  Semantics 

One  reason  for  maintaining  the  simplicity  of  the  bare  PATR- 
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II  formalism  is  to  permit  a  clean  semantics  for  the  language.  We 
have  provided  a  denotational  semantics  for  PATR-II  [Pereira  and 
Shieber,  84]  based  on  the  information  systems  domain  theory  of 
Dana  Scott  [Scott,  82].  Insofar  as  more  complex  formalisms,  such 
as  GPSG  and  LFG,  can  be  modeled  as  appropriate  notations  for 
PATR-II  grammars,  PATR-II’s  denotational  semantics  constitutes 
a  framework  in  which  the  semantics  of  these  formalisms  can  also 
be  defined,  discussed,  and  compared.  As  it  appears  that  not  all  the 
power  of  domain  theory  is  needed  for  the  semantics  of  PATR-II, 
we  are  currently  pursuing  the  possibility  of  building  a  semantics 
based  on  a  less  powerful  model.® 

1.2.4.  Flexibility:  Modeling  Linguistic  Con¬ 
structs 

Clearly,  the  bare  PATR-II  formalism,  as  it  was  presented  in 
Section  1.2.1,  is  sorely  inadequate  for  any  major  attempt  at  build¬ 
ing  natural-language  grammars  because  of  its  verbosity  and  re¬ 
dundancy.  Efficiency  of  encoding  was  temporarily  sacrificed  in  an 
attempt  to  keep  the  underlying  formalism  simple,  general,  and  se¬ 
mantically  well-founded.  However,  given  a  simple  underlying  for¬ 
malism,  we  can  build  more  efficient,  specialized  languages  on  top  of 
it,  much  as  MACLISP  might  be  built  on  top  of  pure  LISP.  And  just 
as  MACLISP  need  not  be  implemented  (and  is  not  implemented) 
directly  in  pure  LISP,  specialized  formalisms  built  conceptually  on 
top  of  pure  PATR-II  need  not  be  so  implemented  (although  cur¬ 
rently  we  do  implement  them  directly  through  pure  PATR-II).  The 
effectiveness  of  this  approach  can  be  seen  in  the  fact  that  at  least  a 
sizable  portion  of  English  syntax  has  been  encoded  in  various  ex¬ 
perimental  PATR-II  grammars  constructed  to  date.  The  syntactic 
constructs  encoded  include  subcategorization  of  various  comple¬ 
ment  types  {NPb,  Sb,  etc.),  active,  passive,  “there”  insertion,  ex- 
traposition,  raising,  and  equi-NP  constructions,  and  unbounded 

*But  see  Pereira  and  Shieber  |84|  tor  arguments  in  favor  of  using  domain  theory 
even  if  all  the  available  power  is  not  utilized. 
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dependencies  (such  as  Wh-movement  and  relative  clauses).  Other 
theory-dependent  devices  that  have  been  modeled  with  PATR-II 
include  head-feature  percolation  [Gazdar  and  Pullum,  82],  and 
LFG-like  semantic  forms  [Kaplan  and  Bresnan,  83).  Note  that 
none  of  these  constructs  and  techniques  required  expansion  of  the 
underlying  formalism;  indeed,  the  constructions  all  make  use  of 
the  techniques  described  in  this  section.  See  Shieber  et  al.  [83]  for 
a  detailed  discussion  of  the  modeling  of  some  of  these  phenomena. 

The  devices  now  available  for  molding  PATR-II  to  conform  to 
a  particular  intended  usage  or  linguistic  theory  are  in  their  nascent 
stage.  However,  because  of  their  great  importance  in  making  the 
PATR-II  system  a  usable  one,  we  will  discuss  them  briefly.  It  is 
important  to  keep  in  mind  that  these  methods  should  not  be  con¬ 
sidered  a  part  of  the  underlying  formalism,  but  merely  “syntactic 
sugar”  to  increase  the  system’s  utility  and  allow  it  to  conform  to 
a  user’s  intentions. 

1. 2.4.1.  Templates 

Because  so  much  of  the  information  in  the  PATR-II  grammars 
under  actual  development  tends  to  be  encoded  in  the  lexicon,  most 
of  our  research  has  been  devoted  to  methods  for  removing  redun¬ 
dancy  in  the  lexicon  by  allowing  the  users  themselves  to  define 
primitive  constructs  and  operations  on  lexical  items.  Primitive 
constructs,  such  as  the  transitive,  dyadic,  or  equi-NP  properties  of 
a  verb,  can  be  defined  by  means  of  templates,  that  is,  DAGs  that 
encode  some  linguistically  isolable  portion  of  the  DAG  of  a  lexical 
item.  These  template  DAGs  can  then  be  combined  to  build  the 
lexical  item  out  of  the  user-defined  primitives. 

As  a  simple  example,  we  could  define  (with  the  following  syn¬ 
tax)  the  template  Verb  as 

Let  Verb  be 

<cat>  =  V 

and  the  template  ThirdSing  as 
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Let  ThirdSing  be 

<agr  number>  =  singular 
<agr  person>  =  third 

The  lexical  entry  for  “knights”  would  then  be 
knights: 

Verb  ThirdSing 

Templates  can  themselves  refer  to  other  templates,  enabling 
definition  of  abstract  linguistic  concepts  hierarchically.  For  in¬ 
stance,  a  modal  verb  template  may  use  an  auxiliary  verb  template, 
which  in  term  may  be  defined  using  the  verb  template  above.  In 
fact,  templates  are  currently  employed  for  abstracting  notions  of 
subcategorization,  verb  form,  semantic  type,  and  a  host  of  other 
concepts. 


1.2. 4.2.  Lexical  Rules 

More  complex  relationships  among  lexical  items  can  be  en¬ 
coded  by  means  of  lexical  rules.  These  rules,  such  as  passive 
and  “there”  insertion,  are  user-definable  operations  on  the  lexical 
items,  enabling  one  variant  of  a  word  to  be  built  from  the  speci¬ 
fication  of  another  variant.  A  lexical  rule  is  specified  as  a  set  of 
selective  unifications  relating  an  input  DAG  and  an  output  DAG. 
Thus,  unification  is  the  primitive  used  in  this  device  as  well. 

Lexical  rules  are  used  to  encode  the  relationships  among  vari¬ 
ous  lexical  entries  that  would  typically  be  thought  of  as  transfor¬ 
mations  or  relation-changing  rules  (depending  on  one’s  ideological 
outlook).  Because  lexical  rules  perform  these  operations,  the  lexi¬ 
con  need  include  only  a  prototype  entry  for  each  verb.  The  variant 
forms  can  be  derived  through  lexical  rules  applied  in  accordance 
with  the  morphology  actually  found  on  the  verb.  (The  morpholog¬ 
ical  analysis  in  the  implementations  of  PATR-II  is  performed  by  a 
program  based  on  the  system  of  Koskenniemi  [83]  and  was  written 
by  Lauri  Karttunen  [83].) 
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For  instance,  given  a  PATR-II  grammar  in  which  the  DAGs  are 
used  to  emulate  the  f-structures  of  LFG,  we  might  write  a  passive 
lexical  rule  as  follows  (following  Bresnan  [83]):® 

Define  Passive  as 

<out  cat>  =  <in  cat> 

<out  form>  =  passprt 
<out  subj>  =  <in  obj> 

<out  obj>  =  <in  subj> 

The  rule  states  in  effect  that  the  output  DAG  (the  one  associ¬ 
ated  with  the  passive  verb  form)  marks  the  lexical  item  as  being 
a  passive  verb  whose  object  is  the  input  DAG’s  subject  and  whose 
subject  is  the  input’s  object.  Such  lexical  rules  have  been  used  for 
encoding  the  active/passive  dichotomy,  “there”  insertion,  extrapo¬ 
sition,  and  other  so-called  relation-changing  rules. 

1.2.5.  Modularity  and  Declarativeness 

The  PATR-II  formalism  is  a  completely  declarative  formal¬ 
ism,  as  evidenced  by  its  denotational  semantics  and  the  order- 
independence  of  its  definition.  Modularity  is  achieved  through 
the  ability  to  define  primitive  templates  and  lexical  rules  that  are 
shared  among  lexical  items,  as  well  as  by  the  declarative  nature 
of  the  grammar  formalism  itself,  removing  problems  of  interaction 
of  rules.  Rules  are  guaranteed  to  always  mean  the  same  thing,  re¬ 
gardless  of  the  environment  of  other  rules  in  which  they  are  placed. 

1.2.6.  Implementability 

Implementability  is  an  empirical  matter,  given  credence  by  the 
fact  that  we  now  have  three  implementations  of  the  formalism. 
One  desirable  aspect  of  the  simplicity  and  declarative  nature  of 
the  formalism  is  that  even  though  the  three  implementations  dif- 
fer  substantially  from  one  another,  using  different  parsing  algo- 

^The  example  is  merely  meant  to  be  indicative  of  the  syntax  for  and  operation 
of  lexical  rules.  We  do  not  present  this  as  a  valid  definition  of  Passive  for  any 
grammar  we  have  written  in  PATR-II. 
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rithms  (with  both  top  down  and  bottona  up  properties),  different 
implementations  of  unification,  different  methods  of  compiling  the 
rules,  all  are  able  to  run  on  exactly  the  same  grammars  yielding 
the  identical  results. 

The  three  implementations  of  the  PATR-II  system  currently  in 
operation  at  SRI  are  as  follows: 

•  An  INTERLISP  version  for  the  DEC-2060  using  a  vari¬ 
ant  of  the  Cocke-Kasami- Younger  parsing  algorithm  and 
the  KIMMO  morphological  analyzer  [Karttunen,  83],  and 
a  limited  programming  environment. 

•  A  ZETALISP  version  for  the  Symbolics  3600  using  a  left- 
corner  parsing  algorithm  and  the  KIMMO  morphological 
analyzer,  with  an  extensive  programming  environment  (due 
primarily  to  Mabry  Tyson)  that  includes  incremental  com¬ 
pilation,  multiple  window  debugging  facilities,  tracing,  and 
an  integrated  editor. 

•  A  Prolog  version  (DEC-10  Prolog)  running  on  the  DEC- 
2060  by  Fernando  Pereira,  designed  primarily  as  a  testbed 
for  experimentation  with  efficient  structure-sharing  DAG 
unification  algorithms,  and  incorporating  an  Earley-style 
parsing  algorithm. 

In  addition,  Lauri  Karttunen  and  his  students  at  the  University 
of  Texas  have  implemented  a  system  baaed  on  PATR-II  but  with 
several  interesting  extensions,  including  disjunction  and  negation 
in  the  graph  structures  [Karttunen,  84].  These  extensions  will  un¬ 
doubtedly  be  integrated  into  the  SRI  systems  and  formal  semantics 
for  them  are  being  pursued. 


1.3.  Conclusion 

The  PATR-II  formalism  was  designed  as  a  computer  language 
for  encoding  linguistic  information.  The  design  was  influenced  by 
current  theory  and  practice  in  computer  science,  and  especially 
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in  the  areas  of  programming  language  design  and  semantics.  The 
formalism  is  simple  (consisting  of  just  one  primitive  operation,  uni¬ 
fication),  powerful  (although  it  can  be  constrained  to  be  decidable), 
mathematically  well-founded  (with  a  complete  denotational  seman¬ 
tics),  flexible  (as  demonstrated  by  its  ability  to  model  analyses  in 
GPSG,  LFG,  DCG  and  other  formalisms),  modular  (because  of  its 
higher-level  notational  devices  such  as  templates  and  lexical  rules), 
declarative  (yielding  order-independence  of  operations),  and  imple- 
mentable  (as  demonstrated  by  three  quite  dissimilar  implemented 
systems  and  one  highly  developed  programming  environment). 

As  we  have  emphasized  herein,  PATR-II  seems  to  represent 
a  convergence  of  techniques  from  several  domains — computer  sci¬ 
ence,  programming  language  design,  natural  language  processing 
and  linguistics.  Its  positioning  at  the  center  of  these  trends  arises, 
however,  not  from  the  admixture  of  many  discrete  techniques,  but 
rather  from  the  application  of  a  single  simple  yet  powerful  concept 
to  the  encoding  of  linguistic  information. 
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Chapter  2. 

Features  and  Values 


This  chapter  was  written  by  Lauri  Karttunen  of  the  University  of 
Texas  at  Austin,  the  Artificial  Intelligence  Center,  SRI  Interna¬ 
tional  and  the  Center  for  the  Study  of  Language  and  Information, 
Stanford  University. 


Abstract 

The  paper  discusses  the  linguistic  aspects  of  a  new  general 
purpose  facility  for  computing  with  features.  The  program  was 
developed  in  connection  with  the  course  I  taught  at  the  University 
of  Texas  in  the  fall  of  1983.  It  is  a  generalized  and  expanded  version 
of  a  system  that  Stuart  Shieber  originally  designed  for  the  PATR- 
II  project  at  SRI  in  the  spring  of  1983  with  later  modifications 
by  Fernando  Pereira  and  me.  Like  its  predecessors,  the  new  Texas 
version  of  the  “DG  (directed  graph)”  package  is  primarily  intended 
for  representing  morphological  and  syntactic  information  but  it 
may  turn  out  to  be  very  useful  for  semantic  representations  too. 


2.1.  Introduction 

Most  schools  of  linguistics  use  some  type  of  feature  notation 
in  their  phonological,  morphological,  syntactic,  and  semantic  de- 
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scriptions.  Although  the  objects  that  appear  in  rules  and  condi¬ 
tions  may  have  atomic  names,  such  as  “k,”  “NP,”  “Subject,”  and 
the  like,  such  high-level  terms  typically  stand  for  collections  of  fea¬ 
tures.  Features,  in  this  sense  of  the  word,  are  usually  thought  of 
as  attribute-value  pairs:  [person:  1st],  [number:  sg],  although  sin¬ 
gleton  features  are  also  admitted  in  some  theories.  The  values  of 
phonological  and  morphological  features  are  traditionally  atomic; 
e.g.  1st,  2nd,  3rd;  they  are  often  binary:  -f,  -.  Most  current  theo¬ 
ries  also  allow  features  that  have  complex  values.  A  complex  value 
is  a  collection  of  features.  For  example,  for  some  languages  it  may 
be  useful  to  postulate  a  feature  called  “agreement”  whose  value  is 
a  feature  bundle  that  specifies  values  for  “number”  and  “person.” 


“ 

rperson : 

Srd]' 

agreement : 

Lnumber : 

sg 

Lexical  Functional  Grammar  (LFG)  [Kaplan  and  Bresnan,  83], 
Unification  Grammar  (UG)  [Kay,  79],  Generalized  Phrase  Struc¬ 
ture  Grammar  (GPSG)  [Gazdar  and  Pullum,  82],  among  others, 
use  complex  features. 

As  shown  above,  features  are  typically  represented  as  matrices. 
Another  way  to  represent  features  is  to  think  of  them  as  directed 
graphs  where  values  correspond  to  nodes  and  attributes  to  vectors: 


agreement 


i 
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In  graphs  of  this  sort,  values  are  reached  by  traversing  paths  of 
attribute  names.  We  use  angle  brackets  to  mark  expressions  that 
designate  paths.  With  that  convention,  the  above  graph  can  also 
be  represented  as  a  set  of  equations: 

<agreement  number>  =  sg 
<agreement  person>  =  3rd 

Such  equations  also  provide  a  convenient  way  to  express  con¬ 
ditions  on  features.  This  idea  lies  at  the  heart  of  UG,  LFG,  and 
the  PATR-II  grammar  for  English  [Shieber,  el  ai,  83]  constructed 
at  SRI.  For  example,  the  equation 

<subject  agreement>  =  <predicate  agreeiiient> 

states  that  subject  and  predicate  have  the  same  value  for  agree¬ 
ment.  In  graph  terms,  this  ci:)rresponds  to  a  lattice  where  two 
vectors  point  to  the  same  node: 


In  a  case  like  this,  the  values  of  the  two  paths  have  been  “uni¬ 
fied.”  To  indicate  unification  in  feature  matrices  one  needs  some 
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convention  such  as  the  connecting  lines  in  LFG-style  functional 
structures  (f-structures)  [Kaplan  and  Bresnan,  83]  to  show  that 
the  values  of  two  attributes  are  the  very  same  entity  and  not  iden¬ 
tical  twins. 

A  third  way  to  view  collections  of  features  is  to  think  of  them 
as  partial  functions  that  assign  values  to  attributes  [Sag  et.ai,  84). 


2.2.  Unification  and  Generalization 

Several  related  grammar  formalisms  (UG,  LFG,  PATR-II,  and 
GPSG)  now  exist  that  are  based  on  a  very  similar  conception  of 
features  and  use  unification  as  their  basic  operation.  In  this  section 
we  survey  briefly  some  properties  of  unification. 

Because  feature  matrices  (lattice  nodes)  are  sets  of  attribute- 
value  pairs,  unification  is  closely  related  to  the  operation  of  forming 
a  union  of  two  sets.  But  there  are  two  important  differences: 

•  unification  can  fail 

•  unification  changes  structure 

While  set  union  is  is  an  operation  that  always  yields  something- 
at  least  the  null  set,  unification  may  fail  to  produce  a  value.  When 
it  fails  the  operands  remain  unchanged;  when  it  succeeds,  the 
operands  are  permanently  altered  in  the  process.  They  become 
the  same  object.  Like  set  union,  unification  is  associative  and 
commutative.  The  result  of  unifying  three  or  more  graphs  in  pairs 
with  pne  another  does  not  depend  on  the  order  in  which  the  oper¬ 
ations  are  performed.  They  all  become  the  same  graph  at  the  end. 
Another  important  characteristic  is  that  unification  can  be  applied 
to  values  that  are  still  indeterminate.  When  a  value  is  eventually 
assigned  to  one  of  two  unified  paths,  the  other  path  acquires  it  at 
the  same  time. 

In  trying  to  unify  two  structures,  A  and  B,  one  proceeds  as 
follows.  If  A  and  B  contain  the  same  attribute  but  have  incom¬ 
patible  values  for  it,  they  cannot  be  unified.  When  A  and  B  are 
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merged,  every  attribute  that  appears  only  in  one  of  the  structures 
is  included  in  (Unify  A  B)  with  its  original  value.  If  some  attribute 
appears  both  in  A  and  B,  then  the  value  of  that  attribute  in  (Unify 
A  B)  is  the  unification  of  the  two  values.  For  example,  if  A  and  B 
are  as  shown  below 


A  ■  (agreesient:  [number:  pi]] 


agreement:  [person:  3rd] 
case:  nominative 


the  effect  of  (Unify  A  B)  is  to  make  them  identical  to 


(Unify  A  B)  “ 


agreement : 


person: 
[number: 
case:  nominative 


3rd' 
pi  - 


Simple  cases  of  grammatical  concord,  such  as  number,  case 
and  gender  agreement  between  determiners  and  nouns  in  many 
languages,  can  be  expressed  straight-forwardly  by  stating  that  the 
values  of  these  features  must  unify. 

Besides  unification,  there  are  other  possible  operations  on  fea¬ 
tures  that  may  have  linguistic  relevance.  Generalization  is  perhaps 
the  the  most  likely  candidate  for  that  status.  It  is  closely  related  to 
set  intersection.  The  generalization  of  two  simple  matrices  A  and 
B  consists  of  the  attribute-value  pairs  that  A  and  B  have  in  com¬ 
mon.  If  A  and  B  have  the  same  attribute  with  different  values,  the 
value  of  that  attribute  in  (Generalize  A  B)  is  the  generalization  of 
the  original  values.  Unlike  unification,  generalization  cannot  fail. 

For  example, 
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A  = 

agreement ; 

'person:  Zndl 
^number:  sg  J 

[casei  noTflipative  J 

B  - 

- 

'gender:  masci 

agreement; 

person:  3rd 
number:  sg  J 

case:  genitive 

(Generalize  A  B)  =  [agreement:  [number;  sg]] 

Generalization  may  to  be  a  useful  notion  for  expressing  how 
number  and  gender  agreement  works  in  coordinate  noun  phrases 
(see  [Gazdar  and  Pullum,  82]).  We  discuss  that  possibility  in  Sec¬ 
tion  2.5  below. 

While  there  is  substantial  agreement  on  what  “unification” 
means,  the  same  is  not  true  of  generalization.  For  example,  it 
is  not  clear  what  the  result  of  generalization  should  be  in  a  case 
where  A  and  B  have  the  same  attribute  with  conflicting  values. 
Is  it  indeterminate  or  a  disjunction  of  the  two  incompatible  val¬ 
ues?  If  generalization  lives  up  to  its  promise  and  turns  out  to  have 
interesting  linguistic  applications,  technical  issues  of  this  sort  can 
perhaps  be  resolved  on  linguistic  grounds. 


2.3.  Limitations  of  Some  Current  For- 

•  malisms 

Most  current  grammar  formalisms  for  features  have  certain 
built-in  limitations.  Three  are  relevant  here: 

•  no  cyclic  structures 

•  no  negation 

•  no  disjunction. 
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The  prohibition  against  cyclicity  rules  out  structures  that  con¬ 
tain  circular  paths,  as  in  the  following  example. 


Here  the  path  <a  b  c>  folds  back  onto  itself,  that  is,  <a>  = 
<a  b  c>.  It  is  not  clear  whether  such  descriptions  should  be  ruled 
out  on  theoretical  grounds.  Whatever  the  case  might  be,  current 
implementations  of  LFG,  UG,  or  GPSG  with  which  I  am  familiar 
do  not  support  them. 

The  prohibition  against  negation  makes  it  impossible  to  char¬ 
acterize  a  feature  by  saying  that  it  does  NOT  have  such  and  such 
a  value.  Except  for  LFG,  none  the  above  theories  allows  speci¬ 
fications  such  as  the  following.  We  use  the  symbol  to  mean 
‘not.’ 

[case;  -dat^ 

.  f”  r number :  so  1 
agreement: 

”  I  [person:  3rdJ 

The  first  statement  says  that  case  is  “not  dative,”  the  sec¬ 
ond  says  that  the  value  of  agreement  is  “anything  but  3rd  person 
singular.”  In  case  of  LFG,  the  formalism  allows  negation  in  func¬ 
tional  descriptions  (f-descriptions)  but  not  in  f-structures  that  are 
derived  from  them. 

Not  allowing  disjunctive  specifications  rules  out  matrices  of  the 
following  sort.  We  indicate  disjunction  by  enclosing  the  alternative 
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values  in  {}. 


r  _ 

renumber ; 

sg  1 

agreement:  < 

[gender : 

femj 

> 

[number: 

Pl]  ^ 

[case:  {nom  acc} 

The  first  line  describes  the  value  of  case  as  being  “either  nom¬ 
inative  or  accusative.”  The  value  for  agreement  is  given  as  “either 
feminine  singular  or  plural.”  Among  the  theories  mentioned  above, 
only  Kay’s  UG  allows  disjunctive  feature  specifications  in  its  for¬ 
malism.  In  LFG,  disjunctions  are  allowed  in  f-descriptions  but 
there  are  no  disjunctive  values  in  f-structures. 

Of  the  three  limitations,  the  first  one  may  be  theoretically 
justified  since  it  has  not  been  shown  that  there  are  phenomena  in 
natural  languages  that  involve  circular  structures  (cf.  [Kaplan  and 
Bresnan,  83],  p.  281).  PATR-II  at  SRI  and  its  expanded  version  at 
tlie  University  of  Texas  allow  such  structures  for  practical  reasons 
because  they  tend  to  arise,  mostly  inadvertently,  in  the  course  of 
grammar  construction  and  testing.  An  implementation  that  does 
not  handle  unification  correctly  in  such  cases  is  too  fragile  to  use. 

The  other  two  restrictions  are  linguistically  unmotivated. 
There  are  many  cases,  especially  in  morphology,  in  which  the  most 
natural  feature  specifications  are  negative  or  disjunctive.  In  fact, 
the  examples  given  above  all  represent  such  cases. 

The  first  example,  [case:  -dat],  arises  in  the  plural  paradigm 
of  words  like  “Kind”  child  in  German.  Such  words  have  two  forms 
in  the  plural:  “Kinder”  and  “Kindern.”  The  latter  is  used  only  in 
the  plural  dative,  the  former  in  the  other  three  cases  (nominative, 
genitive,  accusative).  If  we  accept  the  view  that  there  should  be 
just  one  rather  than  three  entries  for  the  plural  suffix  “-er”,  we 
have  the  choice  between 
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-er  ~  Tnuniber:  pi  1 

[case:  {nom  gen  acc}J 


-er  =  fnumber 
case 


er:  pT 
:  -dat_ 


The  second  alternative  seems  preferrable  given  the  fact  that 
there  is,  in  this  particular  declension,  a  clear  two-way  contrast. 
The  marked  dative  is  in  opposition  with  an  unmarked  form  repre¬ 
senting  all  the  other  cases. 

The  second  example  is  from  English.  Although  the  features 
“number”  and  “person”  are  both  clearly  needed  in  English  verb 
morphology,  most  verbs  are  very  incompletely  specified  for  them. 
In  fact,  the  present  tense  paradigm  of  all  regular  verbs  just  has  two 
forms  of  which  one  represents  the  3rd  person  singular  (“walks”) 
and  the  other  (“walk”)  is  used  for  all  other  persons.  Thus  the 
most  natural  characterization  for  “walk"  is  that  it  is  not  3rd  person 
singular.  The  alternative  is  to  say,  in  effect,  that  “walk”  in  the 
present  tense  has  five  different  interpretations. 

The  system  of  articles  in  German  provides  many  examples  that 
call  for  disjunctive  feature  specifications.  The  article  “die,”  for  ex¬ 
ample,  is  used  in  the  nominative  and  accusative  cases  of  singular 
feminine  nouns  and  all  plural  nouns.  The  entry  given  above  suc¬ 
cinctly  encodes  exactly  this  fact. 

There  are  many  cases  where  disjunctive  specifications  seem 
necessary  for  reasons  other  than  just  descriptive  elegance.  An  item 
which  is  in  underspecified  with  respect  to  some  attribute  can  typi¬ 
cally  appear  in  constructions  that  require  conflicting  values  for  that 
attribute.  For  example,  in  German  [Eisenberg,  73]  noun  phrases 
like: 

des  Dozenten  (gen  sg)  the  docent’s 
der  Dozenten  (gen  pi)  the  docents’. 

can  blend  as  in 
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der  Antrag  des  oder  der  Dozenten 
the  petition  of  the  docent  or  docents. 

This  is  not  possible  when  the  noun  is  overtly  marked  for  number, 
as  in  the  case  of  “des  Professors”  (gen  sg)  and  “der  Professoren” 
(gen  pi): 

*der  Antrag  des  oder  der  Professors 
*der  Antrag  des  oder  der  Professoren 
the  petition  of  the  professor  or  professors 

In  the  light  of  such  cases,  it  seems  reasonable  to  assume  that 
there  is  a  single  form,  “Dozenten,”  which  has  a  disjunctive  feature 
specification,  instead  of  postulating  several  fully  specified,  homony¬ 
mous  lexical  entries.  It  is  obvious  that  the  grammaticality  of  the 
example  crucially  depends  on  the  the  fact  that  “Dozenten”  is  not 
definitely  singular  or  definitely  plural  but  can  be  either. 

Similar  examples  are  easy  to  find  in  many  other  languages, 
arguably,  even  in  English: 

?  this  and  those  sheep 

In  PVench  the  clitic  “me”  is  either  accusative  or  dative.  Because 
of  that,  it  can  simultaniously  function  both  as  direct  and  indirect 
object  of  two  conjoined  verb  phrases,  as  in 

Jean  m’a  frappe  et  donne  des  coups  de  pied 
John  hit  and  kicked  me  (“hit  and  gave  me  kicks”). 

Although  these  examples  provide  convincing  evidence  for  dis¬ 
junctive  feature  values,  they  are  at  the  same  time  extremely  prob¬ 
lematic.  Consider  the  German  case.  Like  “this”  and  “those”  in 
English,  the  two  German  articles  are  in  conflict  with  one  another. 
The  features  of  the  noun  are  compatible  with  either  one  of  the 
two  sets  of  article  features  but  unification  is  not  possible.  The 
operation  would  fail  because  it  in  part  involves  unifying  the  two 
incompatible  sets.  The  only  obvious  alternative  is  to  say  that  the 
merge  of  “Dozenten”  with  “des”  and  “der”  is  done  using  two  iden¬ 
tical  copies  of  the  noun  features.  It  works  technically  but  it  raises 
many  unsettling  general  questions  about  unification. 
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2.4.  Unification  with  Disjunctive  and 
Negative  Feature  Specifications 

I  sketch  here  briefly  how  the  basic  unification  procedure  can 
be  modified  to  deal  with  negation  and  to  admit  disjunctive  values. 
These  ideas  have  been  implemented  in  the  new  Texas  version  of  the 
PATR-II  system  for  features.  (I  am  much  indebted  to  Fernando 
Pereira  for  his  advice  on  this  topic.) 

There  are  no  negative  values  per  ae,  only  negative  constraints. 
A  negative  constraint  attached  to  some  structure  A  limits  the  class 
of  structures  that  can  be  unified  with  A.  It  is  not  part  of  the  con¬ 
tent  of  A  itself.  Negative  constraints  are  created  by  the  following 
operation.  If  A  and  B  are  distinct,  i.e.  contain  a  different  value  for 
some  feature,  then  {Negate  A  B)  does  nothing  to  them.  If  A  and 
B  are  not  distinct,  A  is  marked  with  -B  and  B  with  -A.  These  con¬ 
straints  prevent  the  two  nodes  from  ever  becoming  alike.  When  A 
is  unified  with  C,  unification  succeeds  only  if  the  result  is  distinct 
from  B.  The  result  of  (Unify  A  C)  has  to  satisfy  all  the  negative 
constraints  of  both  A  and  C,  If  inherits  all  negative  constraints  of 
A  and  C  that  could  fail  in  some  later  unification. 

Disjunction  is  more  complicated.  Suppose  A,  B  and  C  are  all 
simple  atomic  values.  In  this  situation  C  unifies  with  { A  B}  just  in 
case  it  is  identical  to  one  or  the  other  of  the  disjuncts.  The  result 
is  C.  Now  suppose  that  A,  B,  and  C  are  all  complex.  Furthermore, 
let  us  suppose  that  A  and  B  are  distinct  but  C  is  compatible  with 
both  of  them  as  in  the  following: 


A  - 


'gender:  fern' 
number:  sg 


B  “  [number:  pi] 

C  =  [case:  acc] 

What  should  be  the  result  of  {Unify  {A  B}  C)?  Because  A  and 
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B  are  incompatible,  we  cannot  actually  unify  C  with  both  of  them. 
That  operation  would  fall.  Because  there  is  no  basis  for  choosing 
one,  both  alternatives  have  to  be  left  open.  Nevertheless,  we  need 
to  take  note  of  the  fact  that  either  A  or  B  is  to  be  unified  with  C. 
This  is  done  by  making  the  result  a  complex  disjunction. 

C*  =  {(A  C)  CB  C)}  . 

The  new  value  of  C,  C’,  is  a  disjunction  of  tuples  which  can 
be,  but  have  not  yet  been  unified.  Tuples  (A  C)  and  (B  C)  are 
sets  of  compatible  structures.  At  least  one  of  the  tuples  in  the 
complex  disjunction  must  always  remain  consistent  regardless  of 
what  happens  to  A,  B,  and  C  later.  After  the  first  unification  we 
can  still  unify  A  with  any  structure  that  it  is  compatible  with,  such 
as: 


D  =  [case:  nora] 

If  this  happens,  then  the  tuple  (A  C)  is  no  longer  consistent. 
A  side  effect  of  A  becoming 


A’ 


gender;  fen 
number:  sg 
case:  nom 


is  that  C’  simultaniously  reduces  to  {(B  C)}.  Since  there  is  now 
only  one  viable  alternative  left,  B  and  C  can  at  this  point  be  uni¬ 
fied.  The  original  result  from  (Unify  {A  B}  C)  now  reduces  to  the 
same 'as  (Unify  B  C). 

C*  '  ■  {(B  C)}  ■  rnuniber:  pll 

(case;  acc  J 


As  the  example  shows,  once  C  is  unified  with  {A  B},  A  and 
B  acquire  a  “positive  constraint.”  All  later  unifications  involv¬ 
ing  them  must  keep  at  least  one  of  the  two  pairs  (A  C),  (B  C) 


28 


unifieable.  If  at  some  later  point  one  of  the  two  tuples  becomes 
inconsistent,  the  members  of  the  sole  remaining  tuple  finally  can 
and  should  be  unified.  When  that  has  happened,  the  positive 
constraint  on  A  and  B  can  also  be  discarded.  A  more  elaborate 
example  of  this  sort  is  given  in  the  Appendix. 

Essentially  the  same  procedure  also  works  for  more  compli¬ 
cated  cases.  For  example,  unification  of  {A  B}  with  {CD}  yields 
{(A  C)  (A  D)  (B  C)  (B  D)}  assuming  that  the  two  values  in  each 
tuple  are  compatible.  Any  pairs  that  could  not  be  unified  are  left 
out.  The  complex  disjunction  is  added  as  a  positive  constraint  to 
all  of  the  values  that  appear  in  it.  The  result  of  unifying  {(A  C) 
(B  C)}  with  {(D  F)  (E  F)}  is  {(A  C  D  F)  (A  C  E  F)  (B  C  D  F)  (B 
C  E  F)},  again  assuming  that  no  alternative  can  initially  be  ruled 
out. 

As  for  generalization,  things  are  considerably  simpler.  The 
result  of  (Generalize  A  B)  inherits  both  negative  and  positive  con¬ 
straints  of  A  and  B.  This  follows  from  the  fact  that  the  general¬ 
ization  of  A  and  B  is  the  maximal  subgraph  of  A  and  B  that  will 
unify  with  either  one  them.  Consequently,  it  is  subject  to  any  con¬ 
straint  that  affects  A  or  B.  This  is  analogous  to  the  fact  that,  in 
set  theory, 

{A-C)r^{B- D)  =  {Ar\B)-{CuD)  . 

In  our  current  implementation,  negative  constraints  are 
dropped  as  soon  as  they  become  redundant  as  far  as  unification 
is  concerned.  For  example,  when  [case:  acc]  is  unified  with  with 
.[case:  -dat],  the  resulting  matrix  is  simply  [case:  acc].  The  neg¬ 
ative  constraint  is  eliminated  since  there  is  no  possibility  that  it 
could  ever  be  violated  later.  This  may  be  a  wrong  policy.  It  has 
to  be  modified  to  make  generalization  work  as  proposed  in  Section 
2.5  for  structures  with  negative  constraints.  If  generalization  is  de¬ 
fined  as  we  have  suggested  above,  negative  constraints  must  always 
be  kept  because  they  never  become  redundant  for  generalization. 

When  negative  or  positive  constraints  are  involved,  unification 
obviously  takes  more  time.  Nevertheless,  the  basic  algorithm  re- 
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mains  pretty  much  the  same.  Allowing  for  constraints  does  not 
significantly  reduce  the  speed  at  which  values  that  do  not  have 
any  get  unified  in  the  Texas  implementation. 

In  the  course  of  working  on  the  project,  I  gained  one  insight 
that  perhaps  should  have  been  obvious  from  the  very  beginning: 
the  problems  that  arise  in  this  connection  are  very  similar  to  those 
that  come  up  in  logic  programming.  One  can  actually  use  the 
feature  system  for  certain  kind  of  inferencing.  For  example,  let 
Mary,  Jane,  and  John  have  the  following  values: 

Mary  =  [hair:  blond] 

Jane  =  [hair;  dark] 

John  “=  [sister:  {Jane  Mary}] 

If  we  now  unify  John  with 

[sister:  [eyes:  b1ue]J 

both  Jane  and  Mary  get  marked  with  the  positive  constraint  that 
at  least  one  of  them  has  blue  eyes.  Suppose  that  we  now  learn  that 
Mary  has  green  eyes.  This  immediately  gives  us  more  information 
about  John  and  Jane  as  well.  Now  we  know  that  Jane’s  eyes  are 
blue  and  that  she  definitely  is  John’s  sister.  The  role  of  positive 
constraints  is  to  keep  track  of  partial  information  in  such  a  way 
that  no  inconsistencies  are  allowed  and  proper  updating  is  done 
when  .more  things  become  known. 


2.5.  Future  prospects:  Agreement  in 
Coordinate  Structures 

One  problem  of  long  standing  for  which  the  present  system 
may  provide  a  simple  solution  is  person  agreement  in  coordinate 
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noun  phrases.  The  conjunction  of  a  1st  person  pronoun  with  either 
2nd  or  3rd  person  pronoun  invariably  yields  1st  person  agreement. 
“I  and  you”  is  equivalent  to  “we,”  as  far  as  agreement  is  concerned. 
When  a  second  person  pronoun  is  conjoined  with  a  third  person 
NP,  the  resulting  conjunction  has  the  agreement  properties  of  a 
second  person  pronoun.  Schematically: 

1st  +  2nd  =  1st  (7  and  you  talked  about  ourselves.) 

3rd  +  1st  =  1st  (She  and  /  talked  about  ourselves.) 

2nd  +  3rd  =  2nd.  ( You  and  he  talked  about  your¬ 
selves.) 

Sag,  Gazdar,  Wasow,  and  Weisler  [84]  propose  a  solution  which 
is  based  on  the  idea  of  deriving  the  person  feature  for  a  coordinate 
noun  phrase  by  generalization  (intersection)  from  the  person  fea¬ 
tures  of  its  heads.  It  is  obvious  that  the  desired  effect  can  be 
obtained  in  any  feature  system  that  uses  the  fewest  features  to 
mark  1st  person,  some  additional  feature  for  2nd  person,  and  yet 
another  for  3rd  person.  Because  generalization  of  1st  and  2nd,  for 
example,  yields  only  the  features  that  two  have  in  common,  the 
one  with  fewest  features  wins. 

Any  such  solution  can  probably  be  implemented  easily  in  the 
framework  outlined  above.  However,  this  proposal  has  one  very 
counterintuitive  aspect:  markedness  hierarchy  is  the  reverse  of 
what  traditionally  has  been  assumed.  Designating  something  as 
3rd  person  requires  the  greatest  number  of  feature  specifications. 
In  the  Sag  et  al.  system,  3rd  person  is  the  most  highly  marked 
member  and  1st  person  the  least  marked  member  of  the  trio.  Tra¬ 
ditionally,  3rd  person  has  been  regarded  as  the  unmarked  case. 

In  our  system,  there  is  a  rather  simple  solution  under  which  the 
value  of  person  feature  in  coordinate  NPs  is  derived  by  generaliza¬ 
tion,  just  as  Sag  et  al.  propose,  which  nevertheless  preserves  the 
traditional  view  of  markedness.  The  desired  result  can  be  obtained 
by  using  negative  constraints  rather  than  additional  features  for  es¬ 
tablishing  a  markedness  hierarchy.  In  addition  to  regular  features 
we  allow  lexical  entries  to  contain  negative  constraints.  The  fol- 
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lowing  feature  specifications  for  personal  pronouns  have  the  effect 
we  are  are  seeking.  First  the  regular  (positiva)  features; 


ist  = 


2nd  ■ 


3rd  = 


conversant;  + 
speaker:  +  J 

conversant;  +1 
speaker:  -  J 

"conversant;  -1 

speaker:  -  J 


The  associated  negative  constraints  are: 


1st  “ 

["conversant ;  -"[[ 

[speaker:  -  JJ 

2nd  - 

[“[conversant: 

3rd  - 

(no  constraints) 

Assuming  that  generalization  with  negative  constraints  works 
as  indicated  above,  i.e.  negative  constraints  are  always  inherited, 
it  immediately  follows  that  the  generalization  of  1st  person  with 
any  other  person  is  compatible  with  only  1st  person  and  that  2nd 
person  wins  over  3rd  when  they  are  combined.  The  results  are  as 
follow's. 
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let  +  2nd  * 


conversant;  - 
”  ["conversant :  -"1 
[speaker;  -  J 


let  +  3rd  •=  [“  fconversant:  -1 

I  [speaker;  -  J 

2nd  +  3rd  =  [speaker;  - 

■[conversant:  -] 


Note  that  the  proper  part  of  lst+2nd  excludes  3rd  person.  It 
is  compatible  with  both  1st  and  2nd  person  but  the  negative  con¬ 
straint  rules  out  the  latter  one.  In  the  case  of  Ist-f  3rd,  the  negative 
constraint  is  compatible  with  1st  person  but  incompatible  with  2nd 
and  3rd.  In  the  last  case,  the  specification  [speaker:  -]  rules  out 
1st  person  and  the  negative  constraint  -[conversant:  -J  eliminates 
3rd  person. 

When  negative  constraints  are  counted  in,  1st  person  is  the 
most  and  3rd  person  the  least  marked  member  of  the  three.  In 
that  respect,  the  proposed  analysis  is  in  line  with  traditional  views 
on  markedness.  Another  relevant  observation  is  that  the  negative 
constraints  on  which  the  result  crucially  depends  are  themselves 
not  too  unnatural.  In  effect,  they  say  of  1st  person  that  it  is 
“neither  2nd  nor  3rd”  and  that  2nd  person  is  “not  3rd.” 

It  will  be  interesting  to  see  whether  other  cases  of  markedness 
can  be  analyzed  in  the  same  way. 
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Appendix  A:  Some  Examples  of  Unifi¬ 
cation 


{These  examples  were  produced  using  the  Texas  version  of  the 
DG  package.) 


case:  {non  acc} 

• 

r 

'gender:  feml 

inf  1 : 

agr :  ^ 

number :  sg  J 

► 

[number:  pi] 

Kinder  = 

case:  [-dat] 

1 

infl; 

[gender : 

neut] 

[number: 

pi 

die  Kinder  ■ 

'case:  {noin  acc} 

infl : 

[gender:  neuti 

[number:  pi  J 

’case : 

acc 

r 

agr : 

fgender:  mascl 

infl:  < 

[number;  sg 

jj 

> 

’case : 

.  dat 

agr : 

V  ^ 

[number:  pi] 

den  Kinder  *  *FAILS* 


den  Kindern  = 

case: 

dat 

“ 

infl ; 

agr : 

gender ; 

neuti 

number; 

PT  J. 

. 
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I  = 


inf  1 : 


case : 

non 

1 

agr : 

number : 

sg  I  1 

person; 

IstJ 

he  = 


’case: 

nom 

- 

inf  1 : 

"gender : 

masc’ 

agr : 

number : 

sg 

person: 

3rd  . 

. 

tense : 

present 

do  = 

infl : 

agr: 

”  ["number: 
[person; 

3?J]. 

"tense:  present 

case:  nom 

infl : 

- 

rnumber:  sg 

'  [person:  lst_ 

he  do  =  +FAILS* 


z 
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(Unify  X  y) 


(Unify  (Unify  x  y)  z) 


36 


Chapter  3. 

The  Semantics  of  Grammar 
Formalisms  Seen  as  Computer 
Languages 

This  chapter  was  written  by  Fernando  C.  N.  Pereira  and  Stuart  M. 
Shieber  of  the  Artificial  Intelligence  Center,  SRI  International  and 
the  Center  for  the  Study  of  Language  and  Information,  Stanford 
University. 


Abstract^ 

The  design,  implementation,  and  use  of  grammar  formalisms 
for  natural  language  have  constituted  a  major  branch  of  computa- 
.tional  linguistics  throughout  its  development.  By  viewing  gram¬ 
mar  formalisms  as  just  a  special  case  of  computer  languages,  we 
can  take  advantage  of  the  machinery  of  denotational  semantics 
to  provide  a  precise  specification  of  their  meaning.  Using  Dana 
Scott’s  domain  theory,  we  elucidate  the  nature  of  the  feature  sys¬ 
tems  used  in  augmented  phrase-structure  grammar  formalisms,  in 
particular  those  of  recent  versions  of  generalized  phrase  structure 

'The  research  reported  Id  this  paper  has  been  made  possible  by  a  gift  from  the 
Systems  Development  Foundation. 
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grammar,  lexical  functional  grammar  and  PATR-II,  and  provide  a 
denotational  semantics  for  a  simple  grammar  formalism.  We  find 
that  the  mathematical  structures  developed  for  this  purpose  con¬ 
tain  an  operation  of  feature  generalization,  not  available  in  those 
grammar  formalisms,  that  can  be  used  to  give  a  partial  account  of 
the  effect  of  coordination  on  syntactic  features. 


3.1.  Introduction 

The  design,  implementation,  and  use  of  grammar  formalisms 
for  natural  language  have  constituted  a  major  branch  of  computa¬ 
tional  linguistics  throughout  its  development.  However,  notwith¬ 
standing  the  obvious  superficial  similarity  between  designing  a 
grammar  formalism  and  designing  a  programming  language,  the 
design  techniques  used  for  grammar  formalisms  have  almost  always 
fallen  short  with  respect  to  those  now  available  for  programming 
language  design. 

Formal  and  computational  linguists  most  often  explain  the 
effect  of  a  grammar  formalism  construct  either  by  example  or 
through  its  actual  operation  in  a  particular  implementation.  Such 
practices  are  frowned  upon  by  most  programming-language  de¬ 
signers;  they  become  even  more  dubious  if  one  considers  that  most 
grammar  formalisms  in  use  are  based  either  on  a  context-free  skele¬ 
ton  with  augmentations  or  on  some  closely  related  device  (such  as 
ATNs),  consequently  making  them  obvious  candidates  for  a  declar¬ 
ative  fcmantic^  extended  in  the  natural  way  from  the  declarative 
semantics  of  context-free  grammars. 

The  last  point  deserves  amplification.  Context-free  grammars 
possess  an  obvious  declarative  semantics  in  which  nonterminals 
represent  sets  of  strings  and  rules  represent  n-ary  relations  over 
strings.  This  is  brought  out  by  the  reinterpretation  familiar  from 

^This  use  of  the  term  “semantics”  should  not  be  confused  with  the  more  common 
usage  denoting  that  portion  of  a  grammar  concerned  with  the  meaning  of  object 
sentences.  Here  we  are  concerned  with  the  meaning  of  the  metalanguage. 
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formal  language  theory  of  context-free  grammars  as  polynomials 
over  concatenation  and  set  union.  The  grammar  formalisms  devel¬ 
oped  from  the  definite-clause  subset  of  first  order  logic  are  the  only 
others  used  in  natural-language  analysis  that  have  been  accorded 
a  rigorous  declarative  semantics — in  this  case  derived  from  the 
declarative  semantics  of  logic  programs  [Colmerauer,  78;  Pereira 
and  Warren,  80;  Pereira,  81]. 

Much  confusion,  wasted  effort,  and  dissension  have  resulted 
from  this  state  of  affairs.  In  the  absence  of  a  rigorous  semantics 
for  a  given  grammar  formalism,  the  user,  critic,  or  implementer  of 
the  formalism  risks  misunderstanding  the  intended  interpretation 
of  a  construct,  and  is  in  a  poor  position  to  compare  it  to  alterna¬ 
tives.  Likewise,  the  inventor  of  a  new  formalism  can  never  be  sure 
of  how  it  compares  with  existing  ones.  As  an  example  of  these 
difficulties,  two  simple  changes  in  the  implementation  of  the  ATN 
formalism,  the  addition  of  a  well-formed  substring  table  and  the 
use  of  a  bottom-up  parsing  strategy,  required  a  rather  subtle  and 
unanticipated  reinterpretation  of  the  register-testing  and  -setting 
actions,  thereby  imparting  a  different  meaning  to  grammars  that 
had  been  developed  for  initial  top-down  backtrack  implementation 
[Woods,  et  ai,  76]. 

Rigorous  definitions  of  grammar  formalisms  can  and  should  be 
made  available.  Looking  at  grammar  formalisms  as  just  a  special 
case  of  computer  languages,  we  can  take  advantage  of  the  ma¬ 
chinery  of  denotationa!  semantics  [Stoy,  77]  to  provide  a  precise 
specification  of  their  meaning.  This  approach  can  elucidate  the 
.  structure  of  the  data  objects  manipulated  by  a  formalism  and  the 
mathematical  relationships  among  various  formalisms,  suggest  new 
possibilities  for  linguistic  analysis  (the  subject  matter  of  the  for¬ 
malisms),  and  establish  connections  between  grammar  formalisms 
and  such  other  fields  of  research  as  programming-language  design 
and  theories  of  abstract  data  types.  This  last  point  is  particularly 
interesting  because  it  opens  up  several  possibilities — among  them 
that  of  imposing  a  type  discipline  on  the  use  of  a  formalism,  with 
all  the  attendant  advantages  of  compile-time  error  checking,  mod- 
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ularity,  and  optimized  compilation  techniques  for  grammar  rules, 
and  that  of  relating  grammar  formalisms  to  other  knowledge  rep¬ 
resentation  languages  [Ait-Kaci,  83], 

As  a  specific  contribution  of  this  study,  we  elucidate  the  na¬ 
ture  of  the  feature  systems  used  in  augmented  phrase-structure 
grammar  formalisms,  in  particular  those  of  recent  versions  of  gen¬ 
eralized  phrase  structure  grammar  (GPSG)  [Gazdar  and  Pullum, 
82;  Sag  et  ai,  84],  lexical-functional  grammar  (LFG)  [Kaplan  and 
Bresnan,  83]  and  PATR-II  [Shieber,  et  ai,  83;  Shieber,  84);  we  find 
that  the  mathematical  structures  developed  for  this  purpose  con¬ 
tain  an  operation  of  feature  generalization,  not  available  in  those 
grammar  formalisms,  that  can  be  used  to  give  a  partial  account  of 
the  effect  of  coordination  on  syntactic  features. 

Just  as  studies  in  the  semantics  of  programming  languages 
start  by  giving  semantics  for  simple  languages,  so  we  will  start 
with  simple  grammar  formalisms  that  capture  the  essence  of  the 
method  without  an  excess  of  obscuring  detail.  The  present  enter¬ 
prise  should  be  contrasted  with  studies  of  the  generative  capacity  of 
formalisms  using  the  techniques  of  formal  language  theory.  First, 
a  precise  definition  of  the  semantics  of  a  formalism  is  a  prerequisite 
for  such  generative-capacity  studies,  and  this  is  precisely  what  we 
are  trying  to  provide.  Second,  generative  capacity  is  a  very  coarse 
gauge:  in  particular,  it  does  not  distinguish  among  different  for¬ 
malisms  with  the  same  generative  capacity  that  may,  however,  have 
very  different  semantic  accounts.  Finally,  the  tools  of  formal  lan¬ 
guage  theory  are  inadequate  to  describe  at  a  sufficiently  abstract 
level  formalisms  that  are  based  on  the  simultaneous  solution  of  sets 
of  constraints  (Kay,  79;  Marcus,  et  ai,  82].  An  abstract  analysis 
of  those  formalisms  requires  a  notion  of  partial  information  that  is 
precisely  captured  by  the  constructs  of  denotational  semantics. 


3.2.  Denotational  Semantics 

In  broad  terms,  denotational  semantics  is  the  study  of  the  con- 
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nection  between  programs  and  mathematical  entities  that  repre¬ 
sent  their  input-output  relations.  For  such  an  account  to  be  useful, 
it  must  be  compositional,  in  the  sense  that  the  meaning  of  a  pro¬ 
gram  is  developed  from  the  meanings  of  its  parts  by  a  fixed  set  of 
mathematical  operations  that  correspond  directly  to  the  ways  in 
which  the  parts  participate  in  the  whole. 

For  the  purposes  of  the  present  work,  denotational  semantics 
will  mean  the  semantic  domain  theory  initiated  by  Scott  and  Stra- 
chey  [Stoy,  77].  In  accordance  with  this  approach,  the  meanings 
of  programming  language  constructs  are  certain  partial  mappings 
between  objects  that  represent  partially  specified  data  objects  or 
partially  defined  states  of  computation.  The  essential  idea  is  that 
the  meaning  of  a  construct  describes  what  information  it  adds  to 
a  partial  description  of  a  data  object  or  of  a  state  of  computation. 
Partial  descriptions  are  used  because  computations  in  general  may 
not  terminate  and  may  therefore  never  produce  a  fully  defined  out¬ 
put,  although  each  individual  step  may  be  adding  more  and  more 
information  to  a  partial  description  of  the  undeliverable  output. 

Domain  theory  is  a  mathematical  theory  of  considerable  com¬ 
plexity.  Potential  nontermination  and  the  use  of  functions  as  “first- 
class  citizens”  in  computer  languages  account  for  a  substantial 
fraction  of  that  complexity.  If,  as  is  the  case  in  the  present  work, 
neither  of  those  two  aspects  comes  into  play,  one  may  be  justified 
in  asking  why  such  a  complex  apparatus  is  used.  Indeed,  both 
the  semantics  of  context-free  grammars  mentioned  earlier  and  the 
semantics  of  logic  grammars  in  general  can  be  formulated  using 
.elementary  set  theory  [Harrison,  78;  van  Emden  and  Kowalski, 
76]. 

However,  using  the  more  complex  machinery  may  be  beneficial 
for  the  following  reasons: 

•  Inherent  partiality,  many  grammar  formalisms  operate  in 
terms  of  constraints  between  elements  that  do  not  fully 
specify  all  the  possible  features  of  an  element. 

•  Technical  economy,  results  that  require  laborious  construc- 
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tions  without  utilizing  domain  theory  can  be  reached  triv¬ 
ially  by  using  standard  results  of  the  theory. 

•  Suggestiveness:  domain  theory  brings  with  it  a  rich  mathe¬ 
matical  structure  that  suggests  useful  operations  one  might 
add  to  a  grammar  formalism. 

•  Extensibiiity:  unlike  a  domain-theoretic  account,  a  special¬ 
ized  semantic  account,  say  in  terms  of  sets,  may  not  be 
easily  extended  as  new  constructs  are  added  to  the  formal¬ 
ism. 

3.3.  The  Domain  of  Feature  Structures 

We  will  start  with  an  abstract  denotational  description  of  a 
simple  feature  system  which  bears  a  close  resemblance  to  the  fea¬ 
ture  systems  of  GPSG,  LFG  and  PATR-II,  although  this  similarity, 
because  of  its  abstractness,  may  not  be  apparent  at  first  glance. 
Such  feature  systems  tend  to  use  data  structures  or  mathematical 
objects  that  are  more  or  less  isomorphic  to  directed  graphs  of  one 
sort  or  another,  or,  as  they  are  sometimes  described,  partial  func¬ 
tions.  Just  what  the  relation  is  between  these  two  ways  of  viewing 
things  will  be  explained  later.  In  general,  these  graph  structures 
are  used  to  encode  linguistic  information  in  the  form  of  attribute- 
value  pairs.  Most  importantly,  partial  information  is  critical  to  the 
use  of  such  systems — for  instance,  in  the  variables  of  definite  clause 
grammars  (Pereira  and  Warren,  80]  and  in  the  GPSG  analysis  of 
coordination  [Sag  ei.al..,  84].  That  is,  the  elements  of  the  feature 
systems,  called  feature  structures  (alternatively,  feature  bundles, 
f-structures  [Kaplan  and  Bresnan,  83],  or  terms)  can  be  partial 
in  some  sense.  The  partial  descriptions,  being  in  a  domain  of  at¬ 
tributes  and  complex  values,  tend  to  be  equational  in  nature:  some 
feature’s  value  is  equated  with  some  other  value.  Partial  descrip¬ 
tions  can  be  understood  in  one  of  two  ways:  either  the  descriptions 
represent  sets  of  fully  specified  elements  of  an  underlying  domain 
or  they  are  regarded  as  participating  in  a  relationship  of  partiality 
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with  respect  to  each  other.  We  will  hold  to  the  latter  view  here. 

What  are  feature  structures  from  this  perspective?  They  are 
repositories  of  information  about  linguistic  entities.  In  domain- 
theoretic  terms,  the  underlying  domain  of  feature  structures  F  is 
a  recursive  domain  of  partial  functions  from  a  set  of  labels  L  (fea¬ 
tures,  attribute  names,  attributes)  to  complex  values  or  primitive 
atomic  values  taken  from  a  set  C  of  constants.  Expressed  formally, 
we  have  the  domain  equation 

F=IL-*F]+C 

The  solution  of  this  domain  equation  can  be  understood  as  a  set 
of  trees  (finite  or  infinite)  with  branches  labeled  by  elements  of  L, 
and  with  other  trees  or  constants  as  nodes.  The  branches  /i, . . . ,  /m 
from  a  node  n  point  to  the  values  n(/i), . . . ,  n(/m)  for  which  the 
node,  as  a  partial  function,  is  defined. 


3.4.  The  Domain  of  Descriptions 


What  the  grammar  formalism  does  is  to  talk  about  F,  not  in 
F.  That  is,  the  grammar  formalism  uses  a  domain  of  descriptions 
of  elements  of  F.  From  an  intuitive  standpoint,  this  is  because, 
for  any  given  phrase,  we  may  know  facts  about  it  that  cannot  be 
encoded  in  the  partial  function  associated  with  it. 

A  partial  description  of  an  element  n  of  F  will  be  a  set  of  equa¬ 
tions  that  constrain  the  values  of  n  on  certain  labels.  In  general, 
•  to  describe  an  element  x  E  F  we  have  equations  of  the  following 
forms: 


(■••(=r{/J)  ••■)(/.•.) 

which  we  prefer  to  write  as 

• •  •  U 
{lu  ■ ■  ■  U 


(•••W/yJ)  •■•)(/,•„) 

Ck 

ihr-hJ 

Cjt 
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with  X  implicit.  The  terms  of  such  equations  are  constants  c  £  C  or 
paths  (/.-j  •  •  •  which  we  identify  in  what  follows  with  strings  in 
L*.  Taken  together,  constants  and  paths  comprise  the  descriptors. 

Using  Scott’s  information  systems  approach  to  domain  con¬ 
struction  [Scott,  82],  we  can  now  build  directly  a  characteriza¬ 
tion  of  feature  structures  in  terms  of  information-bearing  elements, 
equations,  that  engender  a  system  complete  with  notions  of  com¬ 
patibility  and  partiality  of  information. 

The  information  system  D  describing  the  elements  of  F  is  de¬ 
fined,  following  Scott,  as  the  tuple 

D  =  (fJ,  A,Con,l-) 

where  Z?  is  a  set  of  propositions,  Con  is  a  set  of  finite  subsets  of  D, 
the  consistent  subsets,  h  is  an  ent ailment  relation  between  elements 
of  Con  and  elements  of  D  and  A  is  a  special  least  informative 
element  that  gives  no  information  at  all.  We  say  that  a  subset  S  of 
D  is  deductively  closed  if  every  proposition  entailed  by  a  consistent 
subset  of  S  is  in  5.  The  deductive  closure  5  of  5  C  is  the 
smallest  deductively  closed  subset  of  D  that  contains  S. 

The  descriptor  equations  discussed  earlier  are  the  propositions 
of  the  information  system  for  feature  structure  descriptions.  Equa¬ 
tions  express  constraints  among  feature  values  in  a  feature  struc¬ 
ture  and  the  entailment  relation  encodes  the  reflexivity,  symmetry, 
transitivity  and  substitutivity  of  equality.  More  precisely,  we  say 
that  a  finite  set  of  equations  E  entails  an  equation  e  if 

•  Membership'.  eG  E 

•  Reflexivity.  e  is  A  or  =  d  for  some  descriptor  d 

•  Symmetry,  e  is  di  =  d^  and  d^  =  di  is  in  E 

•  Transitivity,  e  is  dt  =  dz  and  there  is  a  descriptor  d  such 
that  di  =  d  and  d  —  d2  are  in  E 

•  Substitutivity.  e  is  di  =  pi  •  ^2  and  both  pt  =  pz  and  di  = 
P2  •  dz  are  in  E 
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•  Iteration:  there  \s  E'  C  E  such  that  E'  e  and  for  all 
e'eE'  E\-^ 

With  this  notion  of  entailment,  the  most  natural  definition  of  the 
set  Con  is  that  a  finite  subset  E  oi  D  \s  consistent  if  and  only 
if  it  does  not  entail  an  inconsistent  equation,  which  has  the  form 
Cl  =  C2,  with  Cl  and  C2  as  distinct  constants. 

An  arbitrary  subset  of  D  is  consistent  if  and  only  if  all  its  finite 
subsets  are  consistent  in  the  way  defined  above.  The  consistent 
and  deductively  closed  subsets  of  D  ordered  by  inclusion  form  a 
complete  partial  order  or  domain  D,  our  domain  of  descriptions  of 
feature  structures. 

Deductive  closure  is  used  to  define  the  elements  of  D  so  that 
elements  defined  by  equivalent  sets  of  equations  are  the  same.  In 
the  rest  of  this  paper,  we  will  specify  elements  of  D  by  convenient 
sets  of  equations,  leaving  the  equations  in  the  closure  implicit. 

The  inclusion  order  C  in  Z?  provides  the  notion  of  a  description 
being  more  or  less  specific  than  another.  The  least-upper-bound 
operation  U  combines  two  descriptions  into  the  least  instantiated 
description  that  satisfies  the  equations  in  both  descriptions,  their 
unification.  The  greatest-lower-bound  operation  n  gives  the  most 
instantiated  description  containing  all  the  equations  common  to 
two  descriptions,  their  generalization. 

The  foregoing  definition  of  consistency  may  seem  very  natural, 
but  it  has  the  technical  disadvantage  that,  in  general,  the  union 
of  two  consistent  sets  is  not  itself  a  consistent  set;  therefore,  the 
corresponding  operation  of  unification  may  not  be  defined  on  cer¬ 
tain  pairs  of  inputs.  Although  this  does  not  cause  problems  at 
this  stage,  it  fails  to  deal  with  the  fact  that  failure  to  unify  is 
not  the  same  as  lack  of  definition  and  causes  technical  difficulties 
when  providing  rule  denotations.  We  therefore  need  a  slightly  less 
natural  definition. 

First  we  add  another  statement  to  the  specification  of  the  en¬ 
tailment  relation: 

•  Falsity,  if  e  is  inconsistent,  {e}  entails  every  element  of  D. 
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That  is,  falsity  entails  anything.  Next  we  define  Con  to  be  simply 
the  set  of  all  finite  subsets  of  D.  The  set  Con  no  longer  corresponds 
to  sets  of  equations  that  are  consistent  in  the  usual  equational 
sense. 

With  the  new  definitions  of  Con  and  h,  the  deductive  closure 
of  a  set  containing  an  inconsistent  equation  is  the  whole  of  D.  The 
partial  order  D  is  now  a  lattice  with  top  element  T  =  P,  and 
the  unification  operation  U  is  always  defined  and  returns  T  on 
unification  failure. 

We  can  now  define  the  description  mapping  d  :  D  F  that 
relates  descriptions  to  the  described  feature  structures.  The  idea 
is  that,  in  proceeding  from  a  description  d  G  D  to  a  feature  struc¬ 
ture  /  G  F,  we  keep  only  definite  information  about  values  and 
discard  information  that  only  states  value  constraints,  but  does 
not  specify  the  values  themselves.  More  precisely,  seeing  d  as  a  set 
of  equations,  we  consider  only  the  subset  [dj  of  d  with  elements  of 
the  form 

(/i  ■■  ■Im)  =  Ck 

Each  e  G  [dJ  defines  an  element  /(e)  of  F  by  the  equations 

mih)  =  h 
!<-x{U)  =  u 

/m  — l(^m)  —  ) 

with  each  of  the  /,•  undefined  for  all  other  labels.  Then,  we  can 
define  5(d)  as 

5(d)  =  U  /(e)  . 

eeLrfj 

This  description  mapping  can  be  shown  to  be  continuous  in  the 
sense  of  domain  theory,  that  is,  it  has  the  properties  that  increasing 
information  in  a  description  leads  to  nondecreasing  information  in 
the  described  structures  (monotoniciiy)  and  that  if  a  sequence  of 
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descriptions  approximates  another  description,  the  same  condition 
holds  for  the  described  structures. 

Note  that  8  may  map  several  elements  of  D  on  to  one  element 
of  F,  For  example,  the  elements  given  by  the  two  sets  of  equations 


i  if  h)  =  (gi)  \ 

J  {fk)  =  c  1 

\  ifh)  =  c  J 

II 

describe  the  same  structure,  because  the  description  mapping  ig¬ 
nores  the  link  between  (/  h)  and  {g  i)  in  the  first  description. 
Such  links  are  useful  only  when  unifying  with  further  descriptive 
elements,  not  in  the  completed  feature  structure,  which  merely 
provides  feature-value  assignments. 

Informally,  we  can  think  of  elements  of  Z)  as  directed  rooted 
graphs  and  of  elements  of  F  as  their  unfoldings  as  trees,  the  un¬ 
folding  being  given  by  the  mapping  8.  It  is  worth  noting  that  if 
a  description  is  cyclic — that  is,  if  it  has  cycles  when  viewed  as  a 
directed  graph — then  the  resulting  feature  tree  will  be  infinite.® 

Stated  more  precisely,  an  element  /  of  a  domain  is  finite,  if 
for  any  ascending  sequence  (d,-)  such  that  /  □  U,  d,-,  there  is  an  j 
such  that  /  □  d,-.  Then  the  cyclic  elements  of  D  are  those  finite 
elements  that  are  mapped  by  8  into  nonfinite  elements  of  F. 


3.5.  Providing  a  Denotation  for  a  Gram¬ 
mar 

We  now  move  on  to  the  question  of  how  the  domain  D  is  used 
to  provide  a  denotational  semantics  for  a  grammar  formalism. 

We  take  a  simple  grammar  formalism  with  rules  consisting  of  a 
context-free  part  over  a  nonterminal  vocabulary  JJ  —  {Ni,. . . ,  Nk} 
and  a  set  of  equations  over  paths  in  ([O..oo]  •  L*)\JC.  A  sample 

®More  precisely  a  rational  tree,  that  is,  a  tree  with  a  finite  number  of  distinct 
subtrees. 
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rule  might  be 


S  NP  VP 

{0  subj)  =  (1) 

{0  predicate)  =  (2) 

{1  agr)  =  {2  agr) 

This  is  a  simplification  of  the  rule  format  used  in  the  PATR-II 
formalism  [Shieber,  et  al.,  83;  Shieber,  84].  The  rule  can  be  read 
as  “an  S  is  an  NP  followed  by  a  VP,  where  the  subject  of  the  S 
is  the  NP,  its  predicate  the  VP,  and  the  ajfreement  of  the  NP  the 
same  as  the  agreement  of  the  VP”. 

More  formally,  a  grammar  is  a  quintuple  G  =  {N ,  S,  L,C,  R), 
where 

•  M  is  a.  finite,  nonempty  set  of  nonterminals  Ni,.. .,  Nk 

•  5  is  the  set  of  strings  over  some  alphabet  (a  flat  domain 
with  an  ancillary  continuous  function  concatenation,  no¬ 
tated  with  the  symbol  •). 

•  R  is  a  set  of  pairs  r  =  (A'^ro  — +  Nr, . . .  Nr„,  Er),  where  Er  is 
a  set  of  equations  between  elements  of  ([0..fn]  •  L*)  \^C. 

As  with  context-free  grammars,  local  ambiguity  of  a  grammar 
means  that  in  general  there  are  several  ways  of  assembling  the  same 
subphrases  into  phrases.  Thus,  the  semantics  of  context-free  gram¬ 
mars  is  given  in  terms  of  sets  of  strings.  The  situation  is  somewhat 
more  complicated  in  our  sample  formalism.  The  objects  specified 
by  the  grammar  are  pairs  of  a  string  and  a  partial  description.  Be¬ 
cause  of  partiality,  the  appropriate  construction  cannot  be  given  in 
terms  of  sets  of  string-description  pairs,  but  rather  in  terms  of  the 
related  domain  construction  of  powerdomains  [Plotkin,  76;  Smyth, 
78;  Scott,  82].  We  will  use  the  Hoare  powerdomain  P  =  Pa{{S  x  D) 
of  the  domain  S  x  D  of  string-description  pairs.  Each  element  of 
P  is  an  approximation  of  a  transduction  relation,  which  is  an  as¬ 
sociation  between  strings  and  their  possible  descriptions. 

We  can  get  a  feeling  for  what  the  domain  P  is  doing  by  exam¬ 
ining  our  notion  of  lexicon.  A  lexicon  will  be  an  element  of  the  do¬ 
main  P*,  associating  with  each  of  the  k  nonterminals  Ni,  I  <  i  <  k 
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a  transduction  relation  from  the  corresponding  coordinate  of  P*. 
Thus,  for  each  nonterminal,  the  lexicon  tells  us  what  phrases  are 
under  that  nonterminal  and  what  possible  descriptions  each  such 
phrase  has.  Here  is  a  sample  lexicon: 

(“Uther”, 

{{agr  num)  =  sg,  {agr  per)  =  3}) 

(“many  knights”, 

{{agr  num)  =  pi,  {agr  per)  =  3}} 

( “storms  Cornwall” , 

{{agr  num)  =  sg}) 

(“sit  at  the  Round  Table”, 

{{agr  num)  =  pi}) 

S:  {} 

By  decomposing  the  effect  of  a  rule  into  appropriate  steps,  we 
can  associate  with  each  rule  r  a  denotation 

[r]  :  p*  — ►  p* 

that  combines  string-description  pairs  by  concatenation  and  uni¬ 
fication  to  build  new  string-description  pairs  for  the  nonterminal 
on  the  left-hand  side  of  the  rule,  leaving  all  other  nonterminals 
untouched.  By  taking  the  union  of  the  denotations  of  the  rules  in 
a  grammar,  (which  is  a  well-defined  and  continuous  powerdomain 
operation,)  we  get  a  mapping 

TaW  =  U  H(«) 

r€R 

from  P*  to  P*  that  represents  a  one-step  application  of  all  the 
rules  of  G  “in  parallel.” 

We  can  now  provide  a  denotation  for  the  entire  grammar  as 
a  mapping  that  completes  a  lexicon  with  all  the  derived  phrases 
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and  their  descriptions.  The  denotation  of  a  grammar  is  the  func¬ 
tion  that  maps  each  lexicon  £  into  the  smallest  fixed  point  of  Tq 
containing  C.  The  fixed  point  is  defined  by 

IClW  =  |j7-i(f)  , 

1=0 


as  Tq  is  continuous. 

It  remains  to  describe  the  decomposition  of  a  rule’s  effect  into 
elementary  steps.  The  main  technicality  to  keep  in  mind  is  that 
rules  state  constraints  among  several  descriptions  (associated  with 
the  parent  and  each  child),  whereas  a  set  of  equations  in  D  con¬ 
strains  but  a  single  description.  This  mismatch  is  solved  by  em¬ 
bedding  the  tuple  {do, . dm)  of  descriptions  in  a  single  larger 
description,  as  expressed  by 

(i)  =  di,  0  <  t  <  m 

and  only  then  applying  the  rule  constraints — now  viewed  as  con¬ 
straining  parts  of  a  single  description.  This  is  done  by  the  indexing 
and  combination  steps  described  below.  The  rest  of  the  work  of 
applying  a  rule,  extracting  the  result,  is  done  by  the  projection  and 
deindexing  steps. 

The  four  steps  for  applying  a  rule 

to  string-description  pairs  (si,  di), . . . ,  (s*,  d*)  are  as  follows.  First, 
we  index  each  dfj  into  d^.  by  replacing  every  path  p  in  any  of  its 
equations  with  the  path  j  •  p.  We  then  combine  these  indexed 
descriptions  with  the  rule  by  unifying  the  deductive  closure  of  Er 
with  all  the  indexed  descriptions: 


d  =  f;,uLJ4 

;=i 
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VVe  can  now  project  d  by  removing  from  it  all  equations  with  paths 
that  do  not  start  with  0.  It  is  clearly  evident  that  the  result  d° 
is  still  deductively  closed.  Finally,  is  deindexed  into  d{^  by 
removing  0  from  the  front  of  all  paths  0  •  p  in  its  equations.  The 
pair  associated  with  Nr^  is  then  (s,,  •  -  •  dr^). 

It  is  not  difficult  to  show  that  the  above  operations  can  be 
lifted  into  operations  over  elements  of  F*  that  leave  untouched  the 
coordinates  not  mentioned  in  the  rule  and  that  the  lifted  operations 
are  continuous  mappings.  With  a  slight  abuse  of  notation,  we  can 
summarize  the  foregoing  discussion  with  the  equation 

IrJ  =  deindex  o  project  o  combine^,  o  index,. 

In  the  case  of  the  sample  lexicon  and  one  rule  grammar  pre¬ 
sented  earlier,  would  be 


NP:  cis  before- • 

VP:  {•••  as  before- • 

(“Uther  storms  Cornwall”, 

{{subj  agr  num)  =  ag,. . .}) 
g  (“many  knights  sit  at  the  Round  Table”, 

{{aubj  agr  num)  =  pi,.. .}} 

(“many  knights  storms  Cornwall”,  T) 


3.6.  Applications 

We  have  used  the  techniques  discussed  here  to  analyze  the  fea¬ 
ture  systems  of  GPSG  [Sag  et.ai,  84],  LFG  [Kaplan  and  Bresnan, 
83]  and  PATR-II  [Shieber,  84].  All  of  them  turn  out  to  be  spe¬ 
cializations  of  our  domain  D  of  descriptions.  Figure  1  provides  a 
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DCG-IP 

PATR-II 

LFG 

GPSG" 

FEATURE  SYSTEM 

full 

finite 

finite 

nonrec. 

CF  SKELETON 

full 

full 

off-line 

full 

“DCGs  based  on  Pro 

og-II  which  allows  cycl 

lie  terms. 

*HPSG,  the  current  Hewlett-Packard  implementation  derived 
from  GPSG,  would  come  more  accurately  under  the  PATR-II 
classification. 

Figure  3.1;  Summary  of  Grammar  System  Properties 


summary  of  two  of  the  most  critical  formal  properties  of  context- 
free-based  grammar  formalisms,  the  domains  of  their  feature  sys¬ 
tems  (full  F,  finite  elements  of  F,  or  elements  of  F  based  on  nonre¬ 
cursive  domain  equations)  and  whether  the  context-free  skeletons 
of  grammars  are  constrained  to  be  off-line  parseable  [Pereira,  83] 
thereby  guaranteeing  decidability. 

Though  notational  differences  and  some  grammatical  devices 
are  glossed  over  here,  the  comparison  is  useful  as  a  first  step  in 
unifying  the  various  formalisms  under  one  semantic  umbrella.  Fur¬ 
thermore,  this  analysis  elicits  the  need  to  distinguish  carefully  be¬ 
tween  the  domain  of  feature  structures  F  and  that  of  descriptions. 
This  distinction  is  not  clear  in  the  published  accounts  of  GPSG 
and  LFG,  which  imprecision  is  responsible  for  a  number  of  uncer¬ 
tainties  in  the  interpretation  of  operators  and  conventions  in  those 
formalisms. 

In  addition  to  formal  insights,  linguistic  insights  have  also  been 
gleaned  from  this  work.  First  of  all,  we  note  that  while  the  systems 
make  crucial  use  of  unification,  generalization  is  also  a  well-defined 
notion  therein  and  might  indeed  be  quite  useful.  In  fact,  it  was 
this  availability  of  the  generalization  operation  that  suggested  a 
simplified  account  of  coordination  facts  in  English  now  being  used 
in  GPSG  [Sag  et.ai,  84]  and  in  an  extension  of  PATR-II  [Kart- 
tunen-,  84].  Though  the  issues  of  coordination  and  agreement  are 
discussed  in  greater  detail  in  these  two  works,  we  present  here  a 
simplified  view  of  the  use  of  generalization  in  a  GPSG  coordination 
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analysis. 

Circa  1982  GPSG  [Gazdar,  et  ai,  82]  analyzed  coordination  by 
using  a  special  principle,  the  conjunct  realization  principle  (CRP), 
to  achieve  partial  instantiation  of  head  features  (including  agree¬ 
ment)  on  the  parent  category.  This  principle,  together  with  the 
head  feature  convention  (HFC)  and  control  agreement  principle 
(CAP),  guaranteed  agreement  between  the  head  noun  of  a  subject 
and  the  head  verb  of  a  predicate  in  English  sentences.  The  HFC, 
in  particular,  can  be  stated  in  our  notation  as  {0  head)  =  {n  head) 
for  n  the  head  of  0. 

A  more  recent  analysis  (Farkas,  et  ai,  83;  Sag,  et  ai,  84]  re¬ 
placed  the  conjunct  realization  principle  with  a  modified  head  fea¬ 
ture  convention  that  required  a  head  to  be  more  instantiated  than 
the  parent,  that  is:  {0  head)  C  (n  head)  for  all  constituents  n 
which  are  heads  of  0.  Making  coordinates  heads  of  their  parent 
achieved  the  effect  of  the  CRP.  Unfortunately,  since  the  HFC  no 
longer  forced  identity  of  agreement,  a  new  principle — the  nomi¬ 
nal  completeness  principle  (NCP),  which  required  that  NP’s  be 
fully  instantiated — was  required  to  guarantee  that  the  appropriate 
agreements  were  maintained. 

Making  use  of  the  order  structure  of  the  domains  we  have  just 
built,  we  can  achieve  straightforwardly  the  effect  of  the  CRP  and 
the  old  HFC  without  any  notion  of  the  NCP.  Our  final  version  of 
the  HFC  merely  requires  that  the  parent’s  head  features  be  the 
generalization  of  the  head  features  of  the  head  children.  Formally, 
we  have: 

{0  head)  =  P  {i  head) 

I'eheads  of  0 

In  the  case  of  parents  with  one  head  child,  this  final  HFC  reduces 
to  the  old  HFC  requiring  identity;  it  reduces  to  the  newer  one,  how¬ 
ever,  in  cases  (like  coordinate  structures)  where  there  are  several 
head  constituents. 

Furthermore,  by  utilizing  an  order  structure  on  the  domain 
of  constants  C,  it  may  be  possible  to  model  that  troublesome  co- 
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ordination  phenomenon,  number  agreement  in  coordinated  noun 
phrases  [Karttunen,  84;  Sag,  et  ai,  84]. 


3.7.  Conclusion 

We  have  approached  the  problem  of  analyzing  the  meaning  of 
grammar  formalisms  by  applying  the  techniques  of  denotational  se¬ 
mantics  taken  from  work  on  the  semantics  of  computer  languages. 
This  has  enabled  us  to 

•  account  rigorously  for  intrinsically  partial  descriptions, 

•  derive  directly  notions  of  unification,  instantiation  and  gen¬ 
eralization, 

•  relate  feature  systems  in  linguistics  with  type  systems  in 
computer  science, 

•  show  that  feature  systems  in  GPSG,  LFG  and  PATR-II  are 
special  cases  of  a  single  construction, 

•  give  semantics  to  a  variety  of  mechanisms  in  grammar  for¬ 
malisms,  and 

•  introduce  operations  for  modeling  linguistic  phenomena 
that  have  not  previously  been  considered. 

We  plan  to  develop  the  approach  further  to  give  accounts  of 
negative  and  disjunctive  constraints  [Karttunen,  84],  besides  the 
simple  equational  constraints  discussed  here. 

On  the  basis  of  these  insights  alone,  it  should  be  clear  that 
the  view  of  grammar  formalisms  as  programming  languages  offers 
considerable  potential  for  investigation.  But,  even  more  impor¬ 
tantly,  the  linguistic  discipline  enforced  by  a  rigorous  approach  to 
the  design  and  analysis  of  grammar  formalisms  may  make  possible 
a  hitherto  unachievable  standard  of  research  in  this  area. 
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